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ABSTRACT 



question; ”What 
individual with 
circumstances?” 
if there was any 



This paper deals with a study designed to pursue 
treatment by whom is most effective for this 
that specific problem under which set of 
One of the objectives of the study was to determine 
relationship between two predictor variables. 



the 



personality type (extraversion-introversion) and perceptual 
orientation (field independence — field dependence) ; and treatment 
effects as shown by four criterion variables: (1) a knowledge test, 

(2) a simulation test, (3) frequency, and (4) variety of career 
exploratory behaviors. A stepwise regression procedure was used to 
analyze the data. No significant relationships were found between 
treatment effects and predictor variables. A variety of difficulties 
which may have contributed to these results were identified and 
implications for future research were presented. (Author) 






ERIC 



<50 052^ EDO 38691 



U.S. OEPARTWENTOF HEALTH, EDUCATION 
& WELFARE 
OFFICE OF EDUCATION 
THIS DOCUMENT HAS BEEN REPRODUCED 
EXACTLY AS RECEIVED FROM THE PERSON OR 
ORGANIZATION ORIGINATING IT, POINTS OF 
VIEW OR OPINIONS STATED DO NOT Nf;CES^ 
SARILY REPRESENT OFFICIAL OFFICE OF EDU' 
CATION POSITION OR POLICY 



Employing Regression Analysis to 
Test Aptitude- Counseling Treatment 
Interactions 



A paper presented at the 
American Educational Research Association 
symposium on effects of aptitude- treatment 
Interaction variables 



March 3, 1970 
Minneapolis, Minnesota 

by 

Bruce Bergland 
Northwestern University 



Introduction 



The need for experimental studies in counseling which use a variety 
of criteria and which contrast competing treatments for different types 
of clients has been increasingly stressed (Kiesler, 1966; Paul, 1967; 

Thoresen, 1969), Indeed main effects of treatments may have little meaning 
in the presence of interactions, and variables not represented in research 
designs have no opportunity to demonstrate their interactive effect 
(Whiteley and Allen, 1969), One long term goal of research is to obtain 
answers to the question, ’Vhat treatment by whom is most effective for 
this individual with that specific problem under which set of circumstances?” 
(Thoresen, 1966), This paper deals with a study designed to pursue this, 
basic question, '■ . 

The study, implemented at Stanford University, explored the differential 
effects of two group counseling treatments on/career exploratory behaviors of 
adolescents. One treatment employed video-taped social models while the 
second treatment was based on structured group interaction using written 
stimulus materials. In an attempt to relate ^ L r* a, Ca ta e 3* s s t o t xr e a, t nxe xx t 

effects two predictor variables, personality type (extraversion-’introversion) 
and perceptual orientation (field independence-field dependence), were included 
In the study. 

The choice of the personality variable of extraversion-introversion was 

based largely on the work of Eysenck (1960; 1967). He argues that for maximum 

effectiveness different counseling procedures should be used with introverts 

and extraverts. According to Eysenck the introvert is responsive to procedures 

, • 

involving considerable reflective and imagery related behaviors while the 
extravert should be involved in group interaction and other procedures based 
on movement and action. 
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The perceptual orientation variable was also chosen because of its 
possible relationship to the treatment effects. Research of Witkin and his 
associates (1962; 1964; 1965; 1967) suggests that perceptual field independence- 
dependence may be important in observational learning. According to Witkin 
who are high in the "ability to perceive objects apart from . . • context" 
(Witkin, 1964, p.l76) are classified as "field-independent" while those low 
in this ability are termed "field dependent," In the present study it was 
hypothesizied that perceptually independent ^s would be more influenced by 
the video-taped models while the perceptually dependent ^s would be more 
Influenced by the structured stimulus materials treatment. 

Method 

• * ' • 1.^ 

Subjects . , ' 

The present study was replicated in three suburban high schools located 
in the vicinity of Stanford University. In each school a standardized 
announcement form regarding an opportunity to take part in a special four 
week career planning program was distributed to all eleventh grade male 

•* I 

students. Students in each school who indicated an interest were initially 
contacted and assessed as to their personality type and perceptual orientation, 
^s were stratified by their scores on these two individual difference variables 
and then randomly assigned to treatment, control and reserve groups* Thirty- 
two _Ss were assigned within each school; -total N = 96. 

Experimental Treatments 

A detailed description of the experimental treatments has been reported 
elsewhere (Thoresen and Hamilton, 1969). Only a brief review will be presented 



here. 



Structured stimulus materials . Planned stimulus materials were used in small 
group counseling sessions consisting of 8 ^s per group, led by one counselor. The 
groups met once a week for four weeks. The written stimulus materials were 
designed to assist adolescents in identifying career goals, gathering relevant and 
reliable information, processing information and considering alternatives. During 
each small group session the counselor verbally and nonverbally reinforced ^ 
responses, such as asking effectively phrased questions relevant to a particular 

I 

occupational characteristic. 

Peer social modeling . Four video-presentfed peer social modeling sessions 
were used in four meetings paralleling the content of the structured stimulus 
materials. In each meeting, ^s observed a video tape in which four peer social 
models and a model counselor discussed specific ways of gathering information 
and processing it according to preferences' and alternatives , On each video 
tape the model counselor verbally and nonverbally reinforced career relevant 
responses on the part of the model students. No group discussion occurred 
after ^s viewed the video tape. 

Criteria . 

The effects of treatment procedures were evaluated by: (1) knowledge 

test of how to obtain and to use relevant and reliable career information, 

(2) simulated career decision situation test where the ^ was asked to imagine 
himself in a hypothetical setting and to list appropriate career exploratory 
behaviors, and (3) frequency and (4) variety of career exploratory behaviors 
of ^s outside tne treatment setting. 
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The first administration of all criterion instruments took place one 
week before treatment sessions began. The second administration of the 
knowledge and the simulation tests occurred one week following the final 
(fourth) treatment sessions. The frequency and variety of career e:Kploratory 
behaviors were assessed for the second time four weeks following the final 
treatment sessions. 




The first criterion variable selected for analysis was the simulation 

test, A tv 70 phase regression analysis was performed (Darlington, 1968). 

First, the post«test simulation scores were regressed on the pre-test 

simulation scores. That portion of a _S's post-test score which deviated from 

what would have been predicted from the pre-test score by linear regression 

was called his residual score. This residual score was considered to contain 

the effects due to treatment. The larger the residual, or 'deviation from 

predicted score, the larger the treatment effect for that individual. The 

♦ 

second phase was to regress a _S's residual score separately on each of his 
predictor variable scores, 

# 

Findings from these regression analyses computed in all^„three schools 
showed that most of the post-test variance was predicted from the pre-test 
scores, leaving little variance to be accounted for by the predictor variable 
scores. The greatest proportion of the residual variance that any one 
predictor variable accounted for was .04, Thus neither personality nor 
perceptual scores of ^s predicted their residual simulation test scores in any 
of the three schools. 

Since no predictive relationship was found between either of the individual 
difference variables and the simulation criterion, it was decided, before 
going further with regression analysis, to compute six two-way analyses of 
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variance (two treatment levels by four levels of ^s: field independent- 

introverted j field independent- extraverted; field dependent- introverted; 
field dependent-extraverted). These tests determined whether there were 
significant interactions between any of the predictor variables and the 
experimental treatments on the reiaaininfi criterion variables: knowledge 

test, frequency and variety of career exploratory behaviors. In these 
analyses, significant interactions served as indicators that regression 
analysis should be performed to determine whether _S differences in percep- 
tual performance or personality type would correspond to differences in 
treatment effects. 

Significant interactions (p< .05) betvzeen the two predictor variables 
and the. criterion variables of frequency and variety of career exploratory 
behaviors were found at one school (Mills), Regression analysis was then 
performed on the peer social modeling and structured stimulus materials • 
treatments at Mills. Each predictor variable was tested separately in 
relation to frequency and variety of career exploratory behaviors. 

The purpose of the regression analysis was to investigate the relation- 
ship between each of the predictor variables and the treatment' effects. To 
carjry out this investigation the analyses were performed in such a way that 
both' the pre-test and perdictor variables were placed in regression equations 
designed to predict post-test scores. The equations have the general form 
$»a4-bxxl+b2x2 where ”Y" is the predicted post-test scores; "a" is a 
constant; b^ is the regression weight for the pre-test; xi is the pre-test 
score; b2 is the regression weight for the predictor variable and x2 is the 
predictor variable score. 
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Results 

The results of the aualyses are presented in Table I in terms of the 
three important parameters of the regression equations, »'a'», "bi" and 
For each regression equation amount of variance in the post- 

test which is accounted for by the pre-test is presented. Finally the 
increase in ’'r 2»' resulting from the addition of the predictor variable to 
the equation is also reported for each equation. The size of £s an 

Indicator of the degree of relationship between pre- and post-test. The 
Increase of R is important because it reflects the amount of criterion 
variance (treatment effect)- accounted for by a particular predictor variable. 



Findings showed that the greatest proportion of the residual variance 
that any one predictor variable accounted for was .09. In general, little 
association was found between either of the predictor variables and residual 
scores obtained from regressing post-test on pre-test scores 



Discussion aqd Implications 



Taken at face value the results indicate little relationship between 
predictor variables and treatment effects. However because of various 
dif f i.culties with some of the procedures used in the study a conclusion of no 
relationship is not warranted at this point. In this section these difficulties ’ 
will be identified and discussed for the purpose of developing suggestions 
for future research. 

^he first difficulty to be discussed concerns sample size. 

A minimum sample size for regression analysis is N-30. As shown in Table I the 
largest sample size for any single regression analysis in this study x/as N=16. 

A sample size of sixteen is not large enough to permit the drawing of reliable 
Inferences concerning the relationships between variables. The small "N", therefore 
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may have contributed significantly to the apparent lack of relationship between 
treatment effects and predictor variables. Subsequent studies should select a 
sample size of at least N^30 if regression analyses are to be performed. 

Nonlinear relationship . A second problem involved the choice of data to 
be used in the regression analysis. In the present study the data for all _Ss 
in a treatment were used in the analyses for that treatment. The difficulty is 
that the interaction between treatment effects and predictor variables may have 
existed only for Ss who scored at the extremes on the • predictor variables « For 
example, an extreme introvert may have been much more responsive to the modeling 
treatment than an extreme extravert while a less extreme introvert and a less 
extreme extravert may have shown no difference in responsiveness. Since scores 
of all Ss were used in the analyses, the data for the less extreme ^s may have 
Confounded relationships present only for extreme ^s. In future studies this 
situation could be dealt with in either of two ways: 1.) In each analysis use 

bn1.y data from Ss scoring at the extremes on the predictor variables, 2.) use 
the data from all ^s but include an X- squared term in the regression equation.^ 
The use of the X-squared term would enable one to assess the strength of this 
curvilinear relationship between predictor variable and treatment effects. 

Pre-treatment variability among Ss . A thitd difficulty is related to the 
manner in which Ss were selected. As explained above volunteers were solicited 
from among eleventh grade male students. Students who volunteered were randomly 
assigned to treatments without attention to initial levels of skills in gathering 
and processing career-relevant information. Consequently each treatment group 
contained some Ss who from the outset were more skilled in information gatheiing 
and processing than other ^s in the same treatment. It is quite posi:.ible that 
these **advanced** Ss showed less change as a result of treatment than ^s with 
lower levels of skills. This is possible because the ’’advanced” Ss had less 



er|c 
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room for improvement than the other Ss. The net result of a lack of change 
for”advanced" would be a tendency to mas!, any relationship betv/een 
predictor variables and treatment effects particularly if the "advanced'' Ss 
had been assessed as extreme on either of the predictor variables. In future 
research a more productive strategy would be to select as ^s only individuals 
who do not show well developed skills on the criterion variables prior to 
treatment. 

"Ceiling effect" of criterion measures . The apparent lack of relationship 
between predictor variables and treatment effects may have been partially due 
to difficulties with criterion variables. There were indications that for two 
variables, the knowledge test and the simulation test, there was a "ceiling 
. effect" which may have masked treatment effects. It was not unusual for 
treatment groups to have an average score of twelve or thirteen on the twenty- 
item knowledge test. For those individuals scoring high on the pretest there 
was little room for improvement in their score. The simulation test may have 
contained a different kind of ceiling. This test contained a description of 
a simulated decision situation. The _S was asked to list such things as 
questions to which he would like to obtain answers, and sources to which he 
would probably go for information. Due to constraints within the school 
setting there was a limit on the time a ^ could spend on the simulation test. 
Consequently, if a _S had worked diligently on the pre-test, it would have been 
difficult for him to show marked gains due to treatment on the posttest largel-y 
because of the time limit. To correct these problems in future studies it would 
be necessary to construct a knowledge test with many more items of varying 
difficulty and to provide with unlimited time to complete the simulation test 
Having ^s record their answers on audio tape instead of writing them, for 
example, might speed up the administration of the test. 
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Hidden figures test . Questions can also be raised concerning the appro- 
priateness of the instrument used to measure perceptual orientation. The 
Hidden Figures Test (French, Ekstrom and Price, 1963 ) used to assess 
perceptual field independence-dependence consisted of two sections of 16 
questions. _Ss were given 10 minutes on each section and were required to find a 
geometric figure, embedded in one of five complex ’’fields,” that corresponded 
to a given design. In retrospect many ^s seemed to have required a ’’warm-up” 
period. For them scores on the second section were much higher than on the 
first. Other ^s failed to complete the test after finding it very difficult 
to detect several of the figures. They invariably received scores of ” 0 ” 
on the second part. In both situations ^s ’ scores reflected the effects of 
learning and motivational variables in addition to their actual performance. 

Future investigations should explore the use of alternative instruments to assess 
perceptual field independGnce-dcpendencc coupled with improved ways cf 
administering instruments to^s. 

Scope of criterion assessment . A sixth difficulty in the present study 
concerns the variety of criterion variables. Two criterion variables measured 
’ knowledge of and ability to use steps in a decision making process. The 
other two criterion variables measured ^s’' information seeking behaviors. 

Did these measures cover all of the relevant variables? In other words was there 
an unmeasured variable which was more responsive to the treatments than those 
variables the study measured? For example, although few treatment effects were 
exhibited by the present criterion variables (Thoresen and Hamilton, 1969 ), it 
is possible that there were important changes in the attitudes which the ^s held 
toward career information gathering and processing. However since these attitudes 
were not measured, it was impossible to relate changes in attitude to predictor 



variables. The implications for future studies are clear: Attention must be 

given to assessing ^s on all variables which may reflect treatment effects and 
which are of interest to the experimenter. 

Treatment objectives . A related problem pertains to the purposes for 
which the experimental treatments were designed. The objectives were to teach 
Ss hox 7 to use the steps in a decision making paradigm and to stimulate them 
to seek information relevant to career choice. These objectives were to be 
achieved in four, fifty-minute treatment sessions,. It is quite possible that 
the objectives were too broad and the sessions too few. In other words by 
attempting to accomplish so much in only four treatment sessions j too little 
treatmeint time and emphasis were devoted to, the accomplishment of any one 
objective. The net result may have been that none of the objectives were 
achieved,, In order to remedy this situation it would be necessary either to 

4 ■ ■ . 

■narrow the focus of the treatments or to provide for many more treatment sessions 
In this way the probability of important treatment effects would be increased. 

■ Correspondingly' there would be~a~'gra’atef""likeTih6dd "of ' finding ah important f elat 
ship between the predictor variables and treatment effects. 

Data analysis procedures . A seventh factor which may have contributed to 
the present lack of results is the type of analysis which was performed. The 
regression analyses in this study dealt separately with each predictor variable 
and each criterion variable. Thus in any one analysis only one criterion 
variable (e.g, frequency of career exploratory behaviors) was regressed on only 
one predictor variable (e.g. extraversion-introversion). Dealing with the 
variables separately assumes that they are in general independent of each other. 
The problem is that although the predictor variables were independent of each 



other, the criterion variables did exhibit important inter correlations. The 

predictor variables correlated r = 0.06. The correlations between criterion 

variables ranged from r = 0.08 to r = 0.96. The knowledge and simulation 

tests tended to correlate r = 0.40. These two tests, however, did not 

correlate with frequency and variety (e.g. r = 0.08). Frequency and variety 

correlated r = 0.96. When criterion variables overlap, an analysis which 

deals with the variables separately tends to waste information. A more 

useful procedure would be one which analyzes the data in terms of sets of 
variables. 

Canonical correlations may have been more appropriate for this study than 
regression analysis, because it does deal with data in terms of sets of 
variables. This procedure differentially weights individual variables so as to 
produce a maximum correlation between linear functions of two sets of variables 
(Cooley and Lohnes, 1962). .If used in the present study, the procedure would 
have developed different weights for each predictor variable and the treatment 

■ produce a maximuia 

correlation between a linear combination of predictor variables and a linear 

combination of criterion variables. Canonical correlation would thereby 
•make use of the overlap among variables to provide a more cSraplete description 
of the relationship between predictor variables and treatment effects. 

ZEtilEl-le^gns. To this point the suggestions for future research have 
dealt with ways in which the present study could be altered so as to increase 
the probability of finding important relationships between predictor variables 
and treatment effects. The final suggestion focuses on a completely different 
design. In the present study single pre- and posttreatment measurements were 
taken on the dependent variables. This procedure may have resulted in much 
important data being missed. For example, what happened during treatment? 
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It is possible that important changes took place at different points while 
the treatments were in progress. The present design provides no information 
relevant to these changes. Future inquiries, • examining interactive effects of 
such variables as subject characteristics, specific treatments, counselor 
characteristics and multiple outcome measures, should use baseline and "during 
treatment" continuous assessment techniques to gather data on how Ss are 
responding through all phases of the study (Thoresen, 1969). This type of 
measurement procedure may require the use of fewer ^s* and/or fewer treatment^ 
groups. However, it will yield more complete information on each S and 
therefore Increase the probability of identifying important relationships. 

Summary 

This paper dealt with a study designed to pursue the question, "What 
treatment by whom is most effective for this individual with that specific 
problem under which set of circumstances?" (Thoresen, 1966). One of the 
objectives of the study was to determine J.f _therej^as j,ny rejl^atiqn^^ 
two predictor variables (personality type: extraversion-introversion and 

I 

perceptual orientation; field independence-field dependence) and treatment 
effects as shown by four criterion variables; 1.) a Knowledge test, 2.) a 
simulation test, 3.) frequency and 4.) variety of career exploratory behaviors 
. A stepwise regression procedure was used to analy 2 je the data. No significant 
relationships were found between treatment effects and predictor variables. A 
variety of difficulties which may have contributed to these results were 
identified and implications for future research were presented. 
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